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Abstract — This paper describes the integration of neural 
network ensembles and interval neutrosophic sets using bagging 
technique for predicting regional-scale potential for mineral 
deposits as well as quantifying uncertainty in the predictions. 
Uncertainty in the types of error and vagueness are considered 
in this paper. Each component in the ensemble consists of 
a pair of neural networks trained for predicting the degrees 
of favourability for deposit and barren. They are considered 
as the truth-membership and the false-membership values, 
respectively. Errors occurred in the prediction are estimated 
using multidimensional scaling and interpolation methods. 
Vagueness is computed as the difference between truth- and 
false-membership values. In this study, uncertainty of type 
vagueness is determined as the indeterminacy-membership 
value. Together these three membership values form an interval 
neutrosophic set. In order to combine and classify outputs 
from components in the ensemble, three aggregation methods 
are proposed in this paper. Our proposed model improves the 
classification performance as compared to the simple majority 
vote and averaging methods. 

Keywords — uncertainty, bagging, feed-forward backpropaga- 
tion neural network, interval neutrosophic set, mineral prospec- 
tivity mapping 

I. Introduction 

In mineral exploration, searching for new mineral deposit 
locations is the most important task. In recent years, neural 
networks were found to give better accuracy result in the 
prediction than other approaches based on the empirical 
statistical methods [1], Nowadays, there are several methods 
for mineral prospectivity prediction based on Geographic 
Information Systems (GIS) and neural networks [1], [2], [3], 
[4]. Although, these methods can predict accurate results, 
uncertainty in the prediction still exists. Hence, assessment 
of uncertainty in mineral prospectivity prediction is one of 
important tasks in mining industry. This paper proposes a 
number of methods for predicting the regional-scale prospec- 
tivity for gold deposits as well as quantifying the uncertainty 
in the prediction. The approaches adopted are based on 
bagging technique applied to ensemble of neural networks 
and interval neutrosophic sets. 

From the experiment conducted in this study, ensemble 
of neural networks was found to give better results and less 
error than a single network [5]. Neural network ensembles 
can be constructed in two steps. First, each neural network 



is trained in an ensemble. Second, the outputs from the 
components in the ensemble are combined. In this paper, a 
bagging technique is applied. Bagging is based on bootstrap 
resampling in which several training sets are created from 
the original training data [6]. The component networks in 
the ensemble are then trained with these data sets. Bagging 
method has been applied in various applications as reported 
in [7], [8], and [9], 

In this paper, we consider two types of uncertainty in 
mineral prospectivity prediction. They are uncertainty of type 
error, and, uncertainty of type vagueness. In general, error 
can result from several sources such as measurement, data 
entry, or processing as well as a lack of knowledge about the 
data [10]. In our study, we concentrate only on errors in the 
process of prediction. Vagueness deals with the concept of 
boundaries which cannot be defined precisely [10]. In [11], 
vague objects can be separated into three types: vague point, 
value line, and vague region in which vague point can be 
defined as a finite set of disjoint sites with known location, 
but the existence of the sites may be uncertain. 

In addition, this study involves gridded map layers in a 
GIS database in which each grid cell represents a site with a 
known location, but the existence of favourability for deposit 
is considered as uncertain. Hence, uncertainty occurs in the 
boundary zone in which the boundary between degree of 
favourability for deposit and degree of favourability for non- 
deposit is not sharp. Some cells may have one hundred 
percent of favourability for deposits. Some cells may have 
zero percent of favourability for mineral deposits and they 
are referred to as non-deposit or barren cells. Most cells have 
degrees of favourability between these two extremes. 

A variety of methods can be used to deal with these two 
types of uncertainties such as stochastic models, probability 
theory, and fuzzy logic [12], [10]. These methods deal 
with different types of uncertainty to different degrees. For 
example, Sunila et al. [ 13] applied a fuzzy model and kriging 
to estimate uncertainty of type vagueness in the boundary 
zone of soil data. Twaroch [14] used intuitionistic fuzzy logic 
to manage uncertainty of regions with vague boundaries. 
In [7], Generalized Regression Neural Networks (GRNN) 
is used to predict errors from known errors produced from 
training neural networks. These errors were then used as 
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dynamically weights in the determination of results from the 
ensemble of neural networks. 

In this paper, ensemble of neural networks based on 
bagging technique is used for mineral prospectivity pre- 
diction. The uncertainty of type error in the prediction is 
estimated using multidimensional scaling and interpolation 
methods. Uncertainty of type vagueness is computed from 
the predicted outputs for each cell on the map. The prediction 
outputs and their uncertainties are represented using an 
interval neutrosophic set and are used to classify the cell 
into either deposit or barren. In this study, three methods 
are proposed for combining and classifying outputs from the 
ensemble. 

The rest of this paper is organized as follows. Section II 
presents the basic theory of interval neutrosophic sets. Sec- 
tion III explains the proposed methods used for the prediction 
and quantification of uncertainties. These proposed meth- 
ods based on ensemble neural networks, bagging, interval 
neutrosophic sets, and interpolation techniques. Section IV 
describes the GIS data set and the results of our experiments. 
Conclusions and future work are presented in Section V. 

II. Interval Neutrosophic Set 

An interval neutrosophic set is generalized from the con- 
cept of a classical set, fuzzy set, interval-valued fuzzy set, 
intuitionistic fuzzy set, interval-valued intuitionistic fuzzy 
set, paraconsistent set, dialetheist set, paradoxist set, and 
tautological set [15]. The membership of an element be- 
longing to the interval neutrosophic set is expressed by 
three values: t, i, and /, which represent truth-membership, 
indeterminacy-membership, and false-membership, respec- 
tively. These membership values can represent several kinds 
of imperfection such as imprecise, incomplete, inconsistent, 
and uncertain information [16]. The three memberships can 
be any real sub-unitary subsets. For example, let A be an 
interval neutrosophic set, then x(85, (20 — 35), (30,40,50)) 
belongs to A means that a; is in A to a degree of 85%, x is 
uncertain to a degree between 20% to 35%, and x is not in A 
to a degree of 30% or 40% or 50%. This research follows the 
definition of interval neutrosophic set that is defined in [15]. 
This definition is described below. 

Let A' be a space of points (objects). An interval neutro- 
sophic set in X is defined as: 



exploration data for grid cell on a map corresponds to a 
deposit, the degree to which the classification of the pattern 
is uncertain, and the degree to which the pattern corresponds 
to a barren cell, respectively. The indeterminacy-membership 
value applied in this study is considered as the degree of 
uncertainty of type vagueness. 

III. Quantification of uncertainty using 

ENSEMBLE OF NEURAL NETWORKS, INTERVAL 
NEUTROSOPHIC SETS, AND INTERPOLATION TECHNIQUES 



Component 1 




Training GIS 
input data layers 




Fig. 1. The proposed prediction model based on the neural networks and 
bagging technique. 



A = {x(T a (x)J a (x),F a (x))\x G X A 
T A : X — > [0, 1] A 
I A : X — * [0, 1] A V 

Pa : X + [ 0 , 1 ]} 

where 

T a is the truth-membership function, 

I A is the indeterminacy-membership function, and 
F a is the false-membership function. 

In this study, an interval neutrosophic set is used to repre- 
sent the result of mineral prospectivity prediction. The truth-, 
indeterminacy-, and false-membership values are interpreted 
to represent the degree to which the pattern of mineral 



In this paper, we apply ensemble neural networks and a 
bagging technique to the mineral prospectivity prediction. 
A bagging technique is used to train neural networks in 
the ensemble. The bagging algorithm uses bootstrap re- 
sampling to generate multiple training sets. In this study, 
each bootstrap sample or bag of data is created by random 
selection of input patterns from the training data set with 
replacement. Each bag contains the same number of training 
patterns as the original data set. Fig.l shows our proposed 
prediction model. Co-register cells in GIS data layers are 
used to create multiple data sets or bags for components in 
the ensemble. Each component in the ensemble consists of 
a pair of neural networks. Both networks use the same bag 



for training. One network is trained to predict degrees of 
favourability for deposit which are interpreted as the truth- 
membership values. The other network is trained to predict 
degrees of favourability for barren which are interpreted as 
the false-membership values. This network is trained with 
the complement of the target output values presented to the 
network used to predict the favourability for deposits. For 
example, if the target output for the deposit network is 1, 
its complement, 0, is used as the target value in the Barren 
network for the same input training pattern. Furthermore, 
errors produced from both networks can be used to estimate 
uncertainty of type error for the unknown or test data set. 



Component 1 




Unknown GIS 
input data layers 




Fig. 2. The proposed prediction and uncertainty model based on the 
integration of interval neutrosophic sets (INS) with neural networks and 
uncertainty estimation. 



Fig. 2 shows our proposed model for the prediction of 
unknown or test data set. A pair of neural networks trained 
for each component in the ensemble is then used for the 
prediction of new data set. We consider the output from each 
component as an interval neutrosophic set. From equation 1, 
we define X 3 as an output from component j, where j = 
1,2,3, ...,m. 

Let Aj be an interval neutrosophic set of X :r Aj can 
be defined as Aj = {x(Ta 3 (x), I a, (a:), Fa 3 (a;))} where 
Ta is the truth (deposit) membership function, I / \ j is the 
indeterminacy membership function, and F,\ :j is the false 
(barren) membership function. 

In this study, we consider the indeterminacy-membership 



value as the degree of uncertainty of type vagueness in 
which the boundary between truth-membership and false- 
membership values cannot be defined precisely. Therefore, 
the indeterminacy-membership value for each cell depends 
on both truth-membership and false-membership values. If 
the difference between these two values is high (e.g. truth 
membership = 1 and false-membership = 0), then the degree 
of uncertainty is low. In contrast, if the difference between 
the two values is low then the degree of uncertainty is 
high (e.g. truth membership = 0.5 and false-membership 
= 0.5). Consequently, for each cell in the map grid, the 
indeterminacy membership value (I a,) can be defined as the 
following. 



I Aj {x) = l-\T Aj {x)-F Aj {x)\. (2) 

The next step is to combine the outputs from all compo- 
nents and then classify the cell into deposit or barren classes. 
In this study, we propose three methods for combining the 
outputs. 

1) Averaging based on equal weight combination. 

In this method, co-register cells in the output lay- 
ers are averaged for the truth-membership and false- 
membership values. After that, the average truth- 
membership and the complement of the average false- 
membership for each cell are combined using a sim- 
ple averaging method. Let T avg (xi) be an average 
truth-membership value for the cell at location i. Let 
h'avgi'J'i) be an average false-membership value for the 
cell at location i. The combined output 0(xi) can be 
computed as the following. 



0(xi) = 



TavgiXi) X (b F aV g(Xi)') 



(3) 



Tavg(.%i') — 



£7=1 t aM) 



(4) 



Favg^Ai) — 



£7=1 F Aj fa) 



m 



(5) 



In order to classify the cell into deposit or barren, we 
apply the threshold value to classify the cell type. A 
range of threshold values ranging from 0.1 to 0.9 in 
steps of 0.1 are created and compared to the output 
O(xi). If the output is greater than the threshold value 
then the cell is classified as a deposit cell. Otherwise, 
the cell is classified as barren. The threshold value that 
can produce the best accuracy in the classification will 
be used in the prediction. In this study, the threshold 
value of 0.5 is the best threshold for this method. 
Uncertainty of type vagueness for each classified cell 
can be computed as the different between the average 
truth-membership and the average false-membership 
values: 1 - | T avg (xi) - F avg (xi ) |. 



2) Majority vote based on equal weight combination. 

In this method, the truth-membership and the com- 
plement of the false-membership values for each cell 
in each output layer are combined using a simple 
averaging method. Let Oj(xi) be the combination 
output for the cell Xi in the output j. Oj(xi) can be 
computed as the following. 



In this study, we consider the certainty for predicting 
the false-membership is equal to the certainty for 
predicting the non false-membership value, which is 
the complement of the false-membership value. Let 
wTj(xi) be the weight for the truth-membership value, 
and wFj(xi) be the weight for the complement of 
the false-membership value, the dynamic combination 
output Oj(xi) can be calculated as follows. 



0j (x,) = + (6) 

A range of threshold values are then compared to 
the result of the combination, Oj(xi). In this study, 
we found that the threshold value that frequently 
produces the best accuracy in the classifications is 
0.5. Hence, we decided to apply the threshold value 
of 0.5 for the classification for all components in 
the ensemble. After that, the majority vote is used 
to make a final classification. If at least half of the 
outputs yield a deposit classification then the cell is 
classified as a deposit. Otherwise, the cell is classified 
as barren. In this method, we consider uncertainty of 
type vagueness for each classified cell as the average 
indeterminacy-membership value from co-register 
cells in the output layers. 



Oj(xi) = (wTj(xi) x T Aj (xi)) + 

(wF'(xi) x (1 - F A .(xi))) 



(7) 



wTj(xi ) 



1 — errTj(xi) 

(1 - errTj(xi )) + (1 - errFj(xi )) 



( 8 ) 



wF'(xi) 



1 — errFj(xi) 

(1 - errTj(xi)) + (1 - errFj(xi)) 



(9) 



Similar to the previous method, a range of threshold 
values are applied and then the majority vote is used 
to classify the cell. Uncertainty of type vagueness 
for each classified cell is the average indeterminacy- 
membership value from co-register cells in the output 
layers. 



3) Majority vote based on dynamic weight combination. 
In this method, uncertainty of type error is also con- 
sidered in the prediction. Fig. 3 shows our proposed 
combination method using uncertainty of type error 
for weighting the combination between the truth- and 
false-membership values. In order to estimate errors in 
the prediction of the truth (deposit) membership values, 
the errors produced from the training truth NN are 
plotted in the multidimensional feature space of the 
training input patterns. After that, a multidimensional 
scaling technique [17], [18] is used to reduce the 
dimension of input feature space to two and then 
applying a Delauney triangulation based interpolation 
method [19] to estimate the uncertainty of type er- 
ror for the new input patterns. Error estimation for 
the prediction of false (barren) membership value is 
also calculated in the same way as the technique 
used for the truth-membership. Let errTj(xi) be an 
estimated uncertainty of type error in the prediction 
of the truth- membership at cell Xi in the output j. 
Let errFj(xi) be an estimated uncertainty of type 
error in the prediction of the false-membership at 
cell Xi in the output j. We determine the weights 
dynamically based on these estimated uncertainties. 
The weight for the truth-membership is computed as 
the complement of the error estimated for the truth- 
membership. The weight for the false-membership is 
calculated as the complement of the error estimated 
for the false-membership. These two types of weight 
are considered as the certainty in the prediction of 
the truth- and false-membership values, respectively. 




Error 

Fig. 3. The proposed combination method based on the integration of 
interval neutrosophic sets (INS) with neural networks as well as error and 
vagueness estimation. 

IV. Experiments 

A. GIS data set 

The study area corresponds to an approximately 100 x 
100 square km region of the Archaean Yilgarn Block, near 
Kalgoorlie, Western Australia. The data set created from this 
area consists of ten layers in raster format. Each Geographic 
Information System (GIS) database layer contains a separate 
variable that is used to help predict the favourability for gold 
deposits. Examples of the variables in these GIS layers are 
solid geology, distance to the nearest fault, and distance to 
the nearest magnetic anomaly. Each GIS layer is divided into 
a grid of square cells of 100 m side. Hence, the map area 
contains 1,254,000 cells. Each cell stores a single attribute 
value which is scaled to a range [0, 1], A training set of 268 
cells were selected for this experiment. For the purposes of 



testing the performance of our predictive system, each cell 
is classified as a deposit or a barren cell. A cell is labeled 
as deposit if it contains greater than 1,000 kg total contained 
gold, otherwise it is classified as a barren cell. Our data set 
consists of 120 deposit cells and 148 barren cells. We use 85 
deposit cells and 102 barren cells for training. The rest; i.e., 
35 deposit cells and 46 barren cells were used for testing. 

B. Experimental methodology and results 

In this study, thirty bags of training data are created for 
thirty components in the ensemble. We use feed-forward 
backpropagation neural networks for both truth NN and 
falsity NN for all components. All networks have the same 
architecture which composes of ten input units, twenty hid- 
den layer units, and a single output unit. All experiments are 
performed using the Matlab software package. The training 
algorithm is traingdx which is based on gradient descent with 
momentum and an adaptive learning rate [20]. However, the 
falsity NN is trained using the complement of the target 
output values used to train the truth NN. Uncertainty of 
type error is also estimated using Matlab. Multidimensional 
scaling is used to reduce the dimensionality of the input 
feature space and then the nearest neighbour interpolation 
based on Delauney triangulation is used to estimate the error. 

In order to combine the output from the components within 
the ensemble, we apply three proposed methods described 
in the previous section. In this paper, we do not consider 
the optimization of the prediction but concentrate only on 
the improvement of the prediction together with the quan- 
tification of uncertainty. The results from our three methods 
are compared in Table I to those obtained by applying two 
existing methods that uses a threshold of 0.5 to only the 
truth-membership of single neural networks instead of a pair 
of networks in the ensemble. The results show that our three 
methods outperform the simple averaging and majority vote 
methods that apply to only the truth-membership value, 

TABLE I 

Classification accuracy for the test data set using neural 

NETWORK ENSEMBLES (A THRESHOLD OF 0.5 IS APPLIED TO EACH 
METHOD). 



Ensemble Method 


Deposit 
% correct 


Barren 
% correct 


All 

% correct 


Averaging using 
equal weight combination 


82.86 


78.26 


80.25 


Majority Vote using 
equal weight combination 


80.00 


76.09 


77.78 


Majority Vote using 
Dynamic weight combination 


80.00 


73.91 


76.54 


Averaging using 
Truth membership 


77.14 


73.91 


75.31 


Majority Vote using 
Truth membership 


80.00 


69.57 


74.07 



Table II shows a selection of the classification results 
for the test set (from a total of 81 input patterns) obtained 
using a majority vote based on equal weighted combination 
(method A), dynamic weighted combination (method B), and 
applying only truth-membership values (method C) together 



with the estimates of uncertainty for method A and B. These 
uncertainty values are the average of the indeterminacy- 
membership values from co-register cells in the output layers. 
For example, the fourth row corresponds to a deposit pattern 
that has been classified correctly as a deposit using our pro- 
posed methods with an uncertainty value of 0.76. However, 
this cell has been classified incorrectly as a barren when 
using the existing majority vote and there is no uncertainty 
information in the classification. The eighth row corresponds 
to a barren pattern which is incorrectly classified as a deposit 
cell with an uncertainty value of 0.82, indicating that the pre- 
diction for the eighth cell involved more uncertainty than the 
prediction for the fourth cell. This uncertainty information 
can support degree of confidence in the prediction. 

TABLE II 

Sample classifications of test set patterns from a majority 

VOTE USING EQUAL WEIGHTED COMBINATION (METHOD A), DYNAMIC 
WEIGHTED COMBINATION (METHOD B), AND APPLYING ONLY 
TRUTH-MEMBERSHIP VALUES (METHOD C). 



Actual 
Cell Type 


Method A 


Method B 


Method C 


Uncertainty 
Value for 
Method A & B 


Deposit 


Deposit 


Deposit 


Deposit 


0.50 


Deposit 


Barren 


Barren 


Barren 


0.74 


Deposit 


Barren 


Barren 


Deposit 


0.79 


Deposit 


Deposit 


Deposit 


Barren 


0.76 


Barren 


Barren 


Deposit 


Barren 


0.77 


Barren 


Barren 


Barren 


Barren 


0.32 


Barren 


Barren 


Barren 


Deposit 


0.68 


Barren 


Deposit 


Deposit 


Deposit 


0.82 



A selection of the classification results for test data set 
using our proposed averaging based on equal weight combi- 
nation are shown in Table III. All cells shown in this table 
are the same cells shown in Table II. A threshold value of 
0.5 is also used in this experiment. Considering the fourth 
row of Table III, the average truth-membership value for this 
cell is 0.44. If we use the existing averaging method using 
only truth- membership value in the classification, this cell 
will be classified incorrectly as a barren. However, if we use 
our proposed averaging based on equal weight combination 
method, this cell is classified correctly as a deposit since 
the combination result used for the classification is 0.57. 
Moreover, our proposed method also has an advantage of 
the ability to represent uncertainty in the prediction in which 
this cell contains uncertainty value of 0.87. If both truth- 
and false-membership values are equal such as the cell in 
the eighth row of this table, then the uncertainty represented 
for this cell is high. This advantage can help users for their 
decision making. 

V. Conclusion and Future work 

This paper has applied a bagging technique and interval 
neutrosophic sets to ensemble of neural networks for mineral 
prospectivity prediction and uncertainty assessment. A pair 
of neural networks was created for each component in the 
ensemble in order to produce degrees of favourability for 
deposit and degrees of favourability for barren cells. These 




TABLE III 

Sample classifications of test set patterns from an equal 

WEIGHTED COMBINATION BETWEEN AVERAGE TRUTH-MEMBERSHIP 
AND AVERAGE FALSE-MEMBERSHIP VALUES. 



Predicted 
Cell Type 


Truth- 

Membership 


False- 

Membership 


Combined 

Value 


Uncertainty 

value 


Deposit 


0.68 


0.20 


0.74 


0.52 


Barren 


0.43 


0.61 


0.41 


0.83 


Deposit 


0.64 


0.63 


0.50 


0.99 


Deposit 


0.44 


0.30 


0.57 


0.87 


Barren 


0.17 


0.19 


0.49 


0.98 


Barren 


0.22 


0.90 


0.16 


0.32 


Barren 


0.47 


0.77 


0.35 


0.69 


Barren 


0.47 


0.48 


0.49 


0.99 



values are corresponding to the truth- and false-membership 
values, respectively. Two types of uncertainty are presented 
in this paper. First, uncertainty of type vagueness represented 
in the form of indeterminacy-membership value is calculated 
as the difference between truth- and false-memberships. 
These three memberships constitute an interval neutrosophic 
set. Second, uncertainties of type error in the prediction of 
truth- and false-memberships are estimated using multidi- 
mensional scaling and interpolation methods. The three pro- 
posed techniques of combing the outputs from components of 
the ensemble outperform simple majority vote and averaging 
methods which are applied only to truth-membership values. 
In this paper, we found that the proposed methods using 
an equal weight combination produces a better accuracy 
results than those using a dynamic weight combination. The 
interpolation technique used to estimate errors for the new 
data set may be the cause of this effect. In future work, 
we plan to apply other interpolation techniques for error 
estimation. We have also planned to apply our methods used 
in this paper to boosting neural networks. 
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